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Purpose 


Provide  Air  Force  perspective  on  MDAP 
Technology  Readiness  Assessments 


Integrity  -  Service  -  Excellence 


2 


Outline 
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> Statutory/Regulatory  Requirement 
>Why  do  a  TRA? 

>AF  TRA  Process 
>What  We’ve  Learned 


Integrity  -  Service  -  Excellence 


3 


What  is  a  TRA? 

(DoD  TRA  Deskbook,  May  2005) 


■  A  TRA  is  an  objective,  systematic,  metrics-based 
process  and  report  that  assesses  the  maturity  of 
Critical  Technology  Elements  (CTEs) 

■  Not  a  risk  assessment;  not  a  design  review 

■  Regulatory  requirement  for  all  acquisition  programs; 
statutory  for  MDAPs 
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Technology  Maturity 

Requirements 


■  Statutory  -  USC  Title  10  Section  2366a  requires 
Milestone  Decision  Authority  (MDA)  Certification 
prior  to  MS/KDP  B  approval  for  Major  Defense 
Acquisition  Programs  (MDAPs) 

■  “the  technology  in  the  program  has  been 
demonstrated  in  a  relevant  environment ’ 

■  Equates  to  Technology  Readiness  Level  (TRL)  6 

■  Regulatory  -  TRAs  required  for  all  programs 

■  DoDI  5000.2:  Required  at  Milestones  (MS)  B  &  C 

■  NSS  Acquisition  Policy  03-01:  Required  at  Key 
Decision  Points  (KDP)  A,  B,  &  C 
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JROC  Technology  Maturity 

Requirement 


■  JROC  Memo  (261-06,  Dec  06)  -  Capability 
Development  Document  (CDD)  and  Capability 
Production  Document  (CPD)  require  discussion  of 
critical  technology  elements  (CTE),  CTE  linkage  to 
Key  Performance  Parameters  (KPP),  and  information 
on  the  Technology  Readiness  Assessment 

■  Purpose:  “to  review  a  program’s  essential 
performance  elements  in  the  context  of  cost, 
schedule  and  technical  risks” 
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Why  do  a  TRA? 


■  GAO  assessments  have  correlated  low 
technology  maturity  with  program  problems 

■  Programs  that  began  development  with 
immature  technologies  averaged 

■  32%  cost  growth 

■  20  months  schedule  growth 

■  Help  acquisition  programs  be  timely,  on  cost, 
meeting  the  user  requirements 

■  Help  acquisition  programs  better  understand 
their  technology  status  &  technical  planning 

■  Provide  senior  leaders  with  current,  accurate 
technical  information  to  make  better  decisions 
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AF  TRA  Process 


CTE  -  Critical  Technology  Element 

IRP  -  Independent  Review  Panel 

TRA  -  Technology  Readiness  Assessment 
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AF  TRA  Process  Variations 


■  Follows  guidance  in  DoD  TRA  Deskbook,  May  2005 

■  Tailored  for  each  MDAP  in  compliance  with  DoD  and  National 
Space  Security  policy  and  guidance 

■  Three  touch  points  to  ensure  objective  TRA 

■  Independent  of  the  program  office 

■  CTEs  assessed  by  an  Independent  Review  Panel 

(IRP)  with  two  basic  variations 

1.  Program  office  builds  initial  TRA  and  briefs  IRP  on  proposed 
CTEs  and  artifacts  on  CTE  maturity  at  formal  IRP  meetings 

2.  IRP  conducts  entire  TRA  (with  program  office  support) 

■  Other  variations 

■  Component  S&T  Executive  provides  the  results  of  the  IRP  to  the 
Independent  Program  Assessment  (IPA)  Team  for  space  systems 
per  NSS  Acquisition  Policy  03-01 

■  Technology  readiness  must  be  addressed  during  source 
selections  conducted  in  conjunction  with  Milestone  B  (or  KDP  B) 
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What  We’ve  Learned 

The  Process 


■  TRA  is  a  process,  not  a  “just  in  time”  milestone 
document 

■  Start  early,  integrate  with  overall  technical  and 
acquisition  planning 

■  Title  10  MDA  certification  requirement  raises  the 
bar  on  TRAs 

■  Need  to  dive  deeper  than  the  component  level  to 
identify  the  technology 

■  A  thorough  &  disciplined  technical  scrub  of  the 
program  is  needed  identify  all  technologies  (from 
which  CTEs  are  determined) 
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What  We’ve  Learned 

IRP  Membership 

■  IRP  membership  needs 

1)  Domain  experienced  experts  who  understand  the 
context  of  the  technology  environment  &  use, 

2)  who  can  connect  the  dots  and  ask  good  questions 
in  a  peer  review  setting,  and 

3)  are  independent  of  the  program  office  and  the 
technologies  being  developed  (sister  service 
participation  adds  bonus  points...) 
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What  We’ve  Learned 

The  Power  of  Change 


■  Programs  will  change  their  approach  if  the  TRA 
shows  maturity  levels  lower  than  expected 

■  However,  they  need  this  information  early  enough 
to  make  changes... 
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What  We’ve  Learned 

Technology  vs.  Design 


■  There  is  a  misconception  between  the  technology 
and  design  implementation 

■  TRL  scale  blurs  pure  technology  with  program 
design  implementation  as  maturity  increases 

■  Can  just  a  technology  be  proven  mature  (TRL  7, 
8,  9)  without  system  integration? 

■  When  does  design  or  technology  change  cross  the 
line  to  become  a  Critical  Technology  Element? 
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What  We’ve  Learned 

Education 


■  Education  of  people  new  to  the  process  needs  to 
start  early 

■  Most  people  have  never  been  hands-on  with  a 
TRA  leading  to  misconceptions 

■  Better  understanding  of  the  TRA  process  and 
methodology  leads  to  efficient  work 

■  Recognize  that  the  broader  workforce  is  still 
climbing  the  learning  curve 


Integrity  -  Service  -  Excellence 


14 


What  We’ve  Learned 

What  it  is  and  is  not 


■  TRLs  are  becoming  very  popular,  but  remember 

■  TRLs  are  only  a  current  snapshot  in  time  -  not  an 
indicator  of  future  success 

■  the  TRA  is  only  an  input  to  program  risk 

■  the  learning  curve  can  be  very  steep  for  those  not 
familiar  -  education  can  make  or  break  a  good 
assessment 

■  Great  tool  for  systems  engineers,  but  most  not 
familiar 
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What  We’ve  Learned 

What  needs  attention 


■  Methodology  lacking  in  some  areas 

■  TRLs  at  the  Systems-of-Systems  (SoS)  level 

■  Defining  environments  for  Space  Systems 

■  Technology  vs  Design  (e.g.,  new  or  novel) 

■  Integrating  technology  maturity  demonstrations  into 
T&E  planning 

■  Demonstrations  are  not  always  part  of  programs' 
"integrated"  V&V  process,  which  is  based  on 
requirements  verification 


Integrity  -  Service  -  Excellence 


16 


Technical  Support  to  America’s  Air  and  Space  Force! 
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